Cost%2DSensitive Semi%2DSupervised Support Vector Machine.

Facial Age Estimation by Learning from Label Distributions.

Multi%2DInstance Dimensionality Reduction.

Multi%2DLabel Learning with Weak Label.

Approximation Stability and Boosting.

On Detecting Clustered Anomalies Using SCiForest.

A New Analysis of Co%2DTraining.
In this paper, we present a new analysis on co-training, a representative paradigm of disagreement-based semi-supervised learning methods. In our analysis the co-training process is viewed as a combinative label propagation over two views; this provides a possibility to bring the graph-based and disagreementbased semi-supervised methods into a unified framework. With the analysis we get some insight that has not been disclosed by previous theoretical studies. In particular, we provide the sufficient and necessary condition for co-training to succeed. We also discuss the relationship to previous theoretical results and give some other interesting implications of our results, such as combination of weight matrices and view split. 1.
A New Analysis of Co%2DTraining.
In this paper, we present a new analysis on co-training, a representative paradigm of disagreement-based semi-supervised learning methods. In our analysis the co-training process is viewed as a combinative label propagation over two views; this provides a possibility to bring the graph-based and disagreementbased semi-supervised methods into a unified framework. With the analysis we get some insight that has not been disclosed by previous theoretical studies. In particular, we provide the sufficient and necessary condition for co-training to succeed. We also discuss the relationship to previous theoretical results and give some other interesting implications of our results, such as combination of weight matrices and view split. 1.
Learning with cost intervals.

Learning with cost intervals.

Multi%2Dinformation Ensemble Diversity.

Multi%2Dmanifold Clustering.
We propose a generative model for high dimensional data consisting of intrinsically low dimensional clusters that are noisily sampled. The proposed model is a mixture of probabilistic principal surfaces (MiPPS) optimized using expectation maximization. We use a Bayesian prior on the model parameters to maximize the corresponding marginal likelihood. We also show empirically that this optimization can be biased towards a good local optimum by using our prior intuition to guide the initialization phase. The proposed unsupervised algorithm naturally handles cases where the data lies on multiple connected components of a single manifold and where the component manifolds intersect. In addition to clustering, we learn a functional model for the underlying structure of each component cluster as a parameterized hyper-surface in ambient noise. This model is used to learn a global embedding that we use for visualization of the entire dataset. We demonstrate the performance of MiPPS in separating and visualizing land cover types in a hyperspectral dataset. 1.
Corrigendum to %22Ensembling neural networks Many could be better than all%22 %5BArtificial Intelligence 137 %281%2D2%29 %282002%29 239%2D263%5D.
Abstract—We present a neural network-based upright frontal face detection system. A retinally connected neural network examines small windows of an image and decides whether each window contains a face. The system arbitrates between multiple networks to improve performance over a single network. We present a straightforward procedure for aligning positive face examples for training. To collect negative examples, we use a bootstrap algorithm, which adds false detections into the training set as training progresses. This eliminates the difficult task of manually selecting nonface training examples, which must be chosen to span the entire space of nonface images. Simple heuristics, such as using the fact that faces rarely overlap in images, can further improve the accuracy. Comparisons with several other state-of-the-art face detection systems are presented, showing that our system has comparable performance in terms of detection and false-positive rates. Index Terms—Face detection, pattern recognition, computer vision, artificial neural networks, machine learning.
S4VM Safe Semi%2DSupervised Support Vector Machine

Multi%2DView Active Learning in the Non%2DRealizable Case

The kth%2C Median and Average Margin Bounds for AdaBoost

Evolutionary Algorithms as Guaranteed Approximation Optimizers

Efficiently Learning Nonlinear Classifiers for Domain Specific Performance Measures

Social Learning.
www.ifpri.org IFPRI’s Discussion Papers contain preliminary material and research results. They have not been subject to formal review by IFPRI’s Publications Review Committee. They are circulated in order to stimulate discussion and critical comment.
Cost%2DSensitive Face Recognition.

Generalized low%2Drank approximations of matrices revisited.

Editorial.
calls &amp; calendarEDITORIAL
Preface.
part of a series of studies in support of the California Governor’s Advisory Committee on Education Excellence. i PREFACE i LIST OF FIGURES iv
Semi%2Dsupervised learning by disagreement.

A framework for modeling positive class expansion with single snapshot.

Advances in Machine Learning
Game-playing applications offer various challenges for machine learning, including opening book learning, learning of evaluation functions, player modeling, and others. In this paper, we briefly highlight the most important achievements of this field and summarize some important recent advance.  
Multilabel dimensionality reduction via dependence maximization.

Advances in Machine Learning
Game-playing applications offer various challenges for machine learning, including opening book learning, learning of evaluation functions, player modeling, and others. In this paper, we briefly highlight the most important achievements of this field and summarize some important recent advance.  


A Framework for Machine Learning with Ambiguous Objects.

A Framework for Machine Learning with Ambiguous Objects.

Enhanced Pictorial Structures for precise eye localization under incontrolled conditions.

Learning a distance metric from multi%2Dinstance multi%2Dlabel data.
Multi-instance multi-label learning (MIML) refers to the learning problems where each example is represented by a bag/collection of instances and is labeled by multiple labels. An example application of MIML is visual object recognition in which each image is represented by multiple key points (i.e., instances) and is assigned to multiple object categories. In this paper, we study the problem of learning a distance metric from multi-instance multi-label data. It is significantly more challenging than the conventional setup of distance metric learning because it is difficult to associate instances in a bag with its assigned class labels. We propose an iterative algorithm for MIML distance metric learning: it first estimates the association between instances in a bag and its assigned class labels, and learns a distance metric from the estimated association by a discriminative analysis; the learned metric will be used to update the association between instances and class labels, which is further used to improve the learning of distance metric. We evaluate the proposed algorithm by the task of automated image annotation, a well known MIML problem. Our empirical study shows an encouraging result when combining the proposed algorithm with citation-kNN, a state-of-theart algorithm for multi-instance learning. 1.
A Convex Method for Locating Regions of Interest with Multi%2Dinstance Learning.
Abstract. In content-based image retrieval (CBIR) and image screening, it is often desirable to locate the regions of interest (ROI) in the images automatically. This can be accomplished with multi-instance learning techniques by treating each image as a bag of instances (regions). Many SVM-based methods are successful in predicting the bag labels, however, few of them can locate the ROIs. Moreover, they are often based on either local search or an EM-style strategy, and may get stuck in local minima easily. In this paper, we propose two convex optimization methods which maximize the margin of concepts via key instance generation at the instance-level and bag-level, respectively. Our formulation can be solved efficiently with a cutting plane algorithm. Experiments show that the proposed methods can effectively locate ROIs, and they also achieve performances competitive with state-of-the-art algorithms on benchmark data sets. 1
Semi%2Dnaive Exploitation of One%2DDependence Estimators.

Least Square Incremental Linear Discriminant Analysis.

Learning instance specific distances using metric propagation.
In many real-world applications, such as image retrieval, it would be natural to measure the distances from one instance to others using instance specific distance which captures the distinctions from the perspective of the concerned instance. However, there is no complete framework for learning instance specific distances since existing methods are incapable of learning such distances for test instance and unlabeled data. In this paper, we propose the Isd method to address this issue. The key of Isd is metric propagation, that is, propagating and adapting metrics of individual labeled examples to individual unlabeled instances. We formulate the problem into a convex optimization framework and derive efficient solutions. Experiments show that Isd can effectively learn instance specific distances for labeled as well as unlabeled instances. The metric propagation scheme can also be used in other scenarios. 1.
Multi%2Dinstance learning by treating instances as non%2DI.I.D. samples.
Previous studies on multi-instance learning typically treated instances in the bags as independently and identically distributed. The instances in a bag, however, are rarely independent in real tasks, and a better performance can be expected if the instances are treated in an non-i.i.d. way that exploits relations among instances. In this paper, we propose two simple yet effective methods. In the first method, we explicitly map every bag to an undirected graph and design a graph kernel for distinguishing the positive and negative bags. In the second method, we implicitly construct graphs by deriving affinity matrices and propose an efficient graph kernel considering the clique information. The effectiveness of the proposed methods are validated by experiments. 1.
Semi%2Dsupervised learning using label mean.
Semi-Supervised Support Vector Machines (S3VMs) typically directly estimate the label assignments for the unlabeled instances. This is often inefficient even with recent advances in the efficient training of the (supervised) SVM. In this paper, we show that S3VMs, with knowledge of the means of the class labels of the unlabeled data, is closely related to the supervised SVM with known labels on all the unlabeled data. This motivates us to first estimate the label means of the unlabeled data. Two versions of the meanS3VM, which work by maximizing the margin between the label means, are proposed. The first one is based on multiple kernel learning, while the second one is based on alternating optimization. Experiments show that both of the proposed algorithms achieve highly competitive and sometimes even the best performance as compared to the state-of-the-art semi-supervised learners. Moreover, they are more efficient than existing S3VMs. 1.
Exploiting Multi%2DModal Interactions A Unified Framework.
Given an imagebase with tagged images, four types of tasks can be executed, i.e., content-based image retrieval, image annotation, text-based image retrieval, and query expansion. For any of these tasks the similarity on the concerned type of objects is essential. In this paper, we propose a framework to tackle these four tasks from a unified view. The essence of the framework is to estimate similarities by exploiting the interactions between objects of different modality. Experiments show that the proposed method can improve similarity estimation, and based on the improved similarity estimation, some simple methods can achieve better performances than some state-of-the-art techniques. 1
Non%2DMetric Label Propagation.
In many applications non-metric distances are better than metric distances in reflecting the perceptual distances of human beings. Previous studies on non-metric distances mainly focused on supervised setting and did not consider the usefulness of unlabeled data. In this paper, we present probably the first study of label propagation on graphs induced from non-metric distances. The challenge here lies in the fact that the triangular inequality does not hold for non-metric distances and therefore, a direct application of existing label propagation methods will lead to inconsistency and conflict. We show that by applying spectrum transformation, non-metric distances can be converted into metric ones, and thus label propagation can be executed. Such methods, however, suffer from the change of original semantic relations. As a main result of this paper, we prove that any non-metric distance matrix can be decomposed into two metric distance matrices containing different information of the data. Based on this recognition, our proposed NMLP method derives two graphs from the original non-metric distance and performs a joint label propagation on the joint graph. Experiments validate the effectiveness of the proposed NMLP method. 1
DrosophilaGene Expression Pattern Annotation through Multi%2DInstance Multi%2DLabel Learning.
The Berkeley Drosophila Genome Project (BDGP) has produced a large number of gene expression patterns, many of which have been annotated textually with anatomical and developmental terms. These terms spatially correspond to local regions of the images; however, they are attached collectively to groups of images, such that it is unknown which term is assigned to which region of which image in the group. This poses a challenge to the development of the computational method to automate the textual description of expression patterns contained in each image. In this paper, we show that the underlying nature of this task matches well with a new machine learning framework, Multi-Instance Multi-Label learning (MIML). We propose a new MIML support vector machine to solve the problems that beset the annotation task. Empirical study shows that the proposed method outperforms the state-of-the-art Drosophila gene expression pattern annotation methods. 1
Drosophila gene expression pattern annotation using sparse features and term%2Dterm interactions.

Selective Ensemble under Regularization Framework.

When Semi%2Dsupervised Learning Meets Ensemble Learning.

Budget Semi%2Dsupervised Learning.

When does Co%2Dtraining Work in Real Data%3F.
Abstract—Co-training, a paradigm of semi-supervised learning, is promised to alleviate effectively the shortage of labeled examples in supervised learning. The standard two-view co-training requires the dataset to be described by two views of features, and previous studies have shown that co-training works well if the two views satisfy the sufficiency and independence assumptions. In practice, however, these two assumptions are often not known or ensured (even when the two views are given). More commonly, most supervised datasets are described by one set of attributes (one view). Thus, they need be split into two views in order to apply the standard twoview co-training. In this paper, we first propose a novel approach to empirically verify the two assumptions of co-training given two views. Then, we design several methods to split single view datasets into two views, in order to make co-training work reliably well. Our empirical results show that, given a whole or a large labeled training set, our view verification and splitting methods are quite effective. Unfortunately, co-training is called for precisely when the labeled training set is small. However, given small labeled training sets, we show that the two co-training assumptions are difficult to verify, and view splitting is unreliable. Our conclusions for co-training’s effectiveness are mixed. If two views are given, and known to satisfy the two assumptions, co-training works well. Otherwise, based on small labeled training sets, verifying the assumptions or splitting single view into two views are unreliable, thus it is uncertain whether the standard co-training would work or not.
Ensemble Learning.
Introduction  When we say we are making a model of a system, we are setting up a tool which can be used to make inferences, predictions and decisions. Each model can be seen as a hypothesis, or explanation, which makes assertions about the quantities which are directly observable and which can only be inferred from their eect on observable quantities.  In the Bayesian framework, knowledge is contained in the conditional probability distributions of the models. We can use Bayes&#039; theorem to evaluate the conditional probability distributions for the unknown parameters, y, given the set of observed quantities, x, using  p (y jx ) =  p (x jy ) p (y)  p (x)  (1) The prior distribution p (y) contains our knowledge of the unknown variables before we make any observ
Boosting.
In an earlier paper, we introduced a new “boosting” algorithm called AdaBoost which, theoretically, can be used to significantly reduce the error of any learning algorithm that consistently generates classifiers whose performance is a little better than random guessing. We also introduced the related notion of a “pseudo-loss ” which is a method for forcing a learning algorithm of multi-label conceptsto concentrate on the labels that are hardest to discriminate. In this paper, we describe experiments we carried out to assess how well AdaBoost with and without pseudo-loss, performs on real learning problems. We performed two sets of experiments. The first set compared boosting to Breiman’s “bagging ” method when used to aggregate various classifiers (including decision trees and single attribute-value tests). We compared the performance of the two methods on a collection of machine-learning benchmarks. In the second set of experiments, we studied in more detail the performance of boosting using a nearest-neighbor classifier on an OCR problem. 
Ensemble.

Multi%2Dinstance clustering with applications to multi%2Dinstance prediction.

A bag%2Dof%2Dwords approach for Drosophila gene expression pattern annotation.

Weak Evolvability Equals Strong Evolvability

Classifier Ensemble with Unlabeled Data

Distributional Features for Text Categorization.

Face recognition under occlusions and variant expressions with partial similarity.

Semi%2Dsupervised document retrieval.

Preface.
part of a series of studies in support of the California Governor’s Advisory Committee on Education Excellence. i PREFACE i LIST OF FIGURES iv
Editorial special issue on selected papers of PAKDD 2007.

Mining extremely small data sets with application to software reuse.

PRICAI 2008 Trends in Artificial Intelligence

Multi%2DLabel Dimensionality Reduction via Dependence Maximization.

Facial age estimation by nonlinear aging pattern subspace.

On the Margin Explanation of Boosting Algorithms.
Much attention has been paid to the theoretical explanation of the empirical success of AdaBoost. The most influential work is the margin theory, which is essentially an upper bound for the generalization error of any voting classifier in terms of the margin distribution over the training data. However, Breiman raised important questions about the margin explanation by developing a boosting algorithm arc-gv that provably generates a larger minimum margin than AdaBoost. He also gave a sharper bound in terms of the minimum margin, and argued that the minimum margin governs the generalization. In experiments however, arc-gv usually performs worse than AdaBoost, putting the margin explanation into serious doubts. In this paper, we try to give a complete answer to Breiman’s critique by proving a bound in terms of a new margin measure called Equilibrium margin (Emargin). The Emargin bound is uniformly sharper than Breiman’s minimum margin bound. This result suggests that the minimum margin is not crucial for the generalization error. We also show that a large Emargin implies good generalization. Experimental results on benchmark datasets demonstrate that AdaBoost usually has a larger Emargin and a smaller test error than arc-gv, which agrees well with our theory. 1
Cost%2Dsensitive face recognition.

Semi%2Dsupervised Learning by Disagreement.

Isolation Forest.
The aim of this study was to assess the impact of isolation on forest bird communities in agricultural land-  scapes in The Netherlands. We studied the avifauna of 235 small (0.1-39 ha) composed of mature deciduous trees in 1985. These were selected in the eastern and central/southern part of the country within 22 regions showing great differences in landscape structure, i.e., degree of isolation. Multiple regression analysis indicated that size was the best single predictor of species number and probability  of occurrence of most species. It turned out that the isolation variables, area of wood, number of woods, interpatch distance, and proximity and density of connecting elements, explained small but significant parts  of the residual variances in species number. No single species was significantly affected by the density of connecting elements. Biogeographical differences between two groups of regions were emphasized. Evidence of  four woodland- species suggested that regional abundance affected the probability of occurrence in small  isolates. 
TEFE A Time%2DEfficient Approach to Feature Extraction.

M3MIML A Maximum Margin Method for Multi%2Dinstance Multi%2Dlabel Learning.

On multi%2Dview active learning and the combination with semi%2Dsupervised learning.
Multi-view learning has become a hot topic during the past few years. In this paper, we first characterize the sample complexity of multi-view active learning. Under the α-expansion assumption, we get an exponential improvement in the sample complexity 1 1 from usual Õ(
On the usefulness of infeasible solutions in evolutionary search A theoretical study.

Mining Bulletin Board Systems Using Community Generation.

A Framework for Modeling Positive Class Expansion with Single Snapshot.

Improving Web search using image snippets.

A new approach to estimating the expected first hitting time of evolutionary algorithms.
www.elsevier.com/locate/artint A new approach to estimating the expected first hitting time
Multi%2DInstance Learning by Treating Instances As Non%2DI.I.D. Samples
Previous studies on multi-instance learning typically treated instances in the bags as independently and identically distributed. The instances in a bag, however, are rarely independent in real tasks, and a better performance can be expected if the instances are treated in an non-i.i.d. way that exploits relations among instances. In this paper, we propose two simple yet effective methods. In the first method, we explicitly map every bag to an undirected graph and design a graph kernel for distinguishing the positive and negative bags. In the second method, we implicitly construct graphs by deriving affinity matrices and propose an efficient graph kernel considering the clique information. The effectiveness of the proposed methods are validated by experiments. 1.
Correction to %22Automatic Age Estimation Based on Facial Aging Patterns%22.
We describe a method for the automatic acquisition of the hyponymy lexical relation from unrestricted text. Two goals motivate the approach: (i) avoidance of the need for pre-encoded knowledge and (ii) applicability across a wide range of text. We identify a set of lexicosyntactic patterns that are easily recognizable, that occur frequently and across text genre boundaries, and that indisputably indicate the lexical relation of interest. We describe a method for discovering these patterns and suggest that other lexical relations will also be acquirable in this way. A subset of the acquisition algorithm is implemented and the results are used to augment and critique the structure of a large hand-built thesaurus. Extensions and applications to areas such as information retrieval are suggested. 1 Introduction  Currently there is much interest in the automatic acquisition of lexical syntax and semantics, with the goal of building up large lexicons for natural language processing. Projects ...
MIML A Framework for Learning with Ambiguous Objects

Correction to %22Automatic Age Estimation Based on Facial Aging Patterns%22.
We describe a method for the automatic acquisition of the hyponymy lexical relation from unrestricted text. Two goals motivate the approach: (i) avoidance of the need for pre-encoded knowledge and (ii) applicability across a wide range of text. We identify a set of lexicosyntactic patterns that are easily recognizable, that occur frequently and across text genre boundaries, and that indisputably indicate the lexical relation of interest. We describe a method for discovering these patterns and suggest that other lexical relations will also be acquirable in this way. A subset of the acquisition algorithm is implemented and the results are used to augment and critique the structure of a large hand-built thesaurus. Extensions and applications to areas such as information retrieval are suggested. 1 Introduction  Currently there is much interest in the automatic acquisition of lexical syntax and semantics, with the goal of building up large lexicons for natural language processing. Projects ...
Individual Stable Space An Approach to Face Recognition Under Uncontrolled Conditions.

Spectrum of Variable%2DRandom Trees.

Top 10 algorithms in data mining.
Abstract This paper presents the top 10 data mining algorithms identified by the IEEE International Conference on Data Mining (ICDM) in December 2006: C4.5, k-Means, SVM, Apriori, EM, PageRank, AdaBoost, kNN, Naive Bayes, and CART. These top 10 algorithms are among the most influential data mining algorithms in the research community. With each algorithm, we provide a description of the algorithm, discuss the impact of the algorithm, and review current and further research on the algorithm. These 10 algorithms cover classification,
Constraint Score A new filter method for feature selection with pairwise constraints.

Advances in Knowledge Discovery and Data Mining

Emerging Technologies in Knowledge Discovery and Data Mining

Multi%2DLabel Learning by Instance Differentiation.

Semi%2DSupervised Learning with Very Few Labeled Training Examples.
In semi-supervised learning, a number of labeled examples are usually required for training an initial weakly useful predictor which is in turn used for exploiting the unlabeled examples. However, in many real-world applications there may exist very few labeled training examples, which makes the weakly useful predictor difficult to generate, and therefore these semisupervised learning methods cannot be applied. This paper proposes a method working under a two-view setting. By taking advantages of the correlations between the views using canonical component analysis, the proposed method can perform semi-supervised learning with only one labeled training example. Experiments and an application to content-based image retrieval validate the effectiveness of the proposed method.
Mining Ambiguous Data with Multi%2Dinstance Multi%2Dlabel Representation.

Single Image Subspace for Face Recognition.
Abstract. Small sample size and severe facial variation are two challenging problems for face recognition. In this paper, we propose the SIS (Single Image Subspace) approach to address these two problems. To deal with the former one, we represent each single image as a subspace spanned by its synthesized (shifted) samples, and employ a newly designed subspace distance metric to measure the distance of subspaces. To deal with the latter one, we divide a face image into several regions, compute the contribution scores of the training samples based on the extracted subspaces in each region, and aggregate the scores of all the regions to yield the ultimate recognition result. Experiments on well-known face databases such as AR, Extended YALE and FERET show that the proposed approach outperforms some renowned methods not only in the scenario of one training sample per person, but also in the scenario of multiple training samples per person with significant facial variations. 1
Structure Learning of Probabilistic Relational Models from Incomplete Relational Data.

Analyzing Co%2Dtraining Style Algorithms.
Abstract. Co-training is a semi-supervised learning paradigm which trains two learners respectively from two different views and lets the learners label some unlabeled examples for each other. In this paper, we present a new PAC analysis on co-training style algorithms. We show that the co-training process can succeed even without two views, given that the two learners have large difference, which explains the success of some co-training style algorithms that do not require two views. Moreover, we theoretically explain that why the co-training process could not improve the performance further after a number of rounds, and present a rough estimation on the appropriate round to terminate co-training to avoid some wasteful learning rounds. 1
Cocktail Ensemble for Regression.

On the relation between multi%2Dinstance learning and semi%2Dsupervised learning.
Multi-instance learning and semi-supervised learning are different branches of machine learning. The former attempts to learn from a training set consists of labeled bags each containing many unlabeled instances; the latter tries to exploit abundant unlabeled instances when learning with a small number of labeled examples. In this paper, we establish a bridge between these two branches by showing that multi-instance learning can be viewed as a special case of semi-supervised learning. Based on this recognition, we propose the MissSVM algorithm which addresses multi-instance learning using a special semisupervised support vector machine. Experiments show that solving multi-instance problems from the view of semi-supervised learning is feasible, and the MissSVM algorithm is competitive with state-of-the-art multiinstance learning algorithms. 1.
Exploiting Image Contents in Web Search.

Semi%2DSupervised Dimensionality Reduction.
Dimensionality reduction is among the keys in mining highdimensional data. This paper studies semi-supervised dimensionality reduction. In this setting, besides abundant unlabeled examples, domain knowledge in the form of pairwise constraints are available, which specifies whether a pair of instances belong to the same class (must-link constraints) or different classes (cannot-link constraints). We propose the SSDR algorithm, which can preserve the intrinsic structure of the unlabeled data as well as both the must-link and cannot-link constraints defined on the labeled examples in the projected low-dimensional space. The SSDR algorithm is efficient and has a closed form solution. Experiments on a broad range of data sets show that SSDR is superior to many established dimensionality reduction methods. 1
Guest Editorial to the Special Issue on Advances of Artificial Intelligence and Knowledge Engineering in China.

Semisupervised Regression with Cotraining%2DStyle Algorithms.

Automatic Age Estimation Based on Facial Aging Patterns.
While recognition of most facial variations, such as identity, expression and gender, has been exten-sively studied, automatic age estimation has rarely been explored. In contrast to other facial variations, aging variation presents several unique characteristics which make age estimation a challenging task. This paper proposes an automatic age estimation method named AGES (AGing pattErn Subspace). The basic idea is to model the aging pattern, which is defined as the sequence of a particular individual’s face images sorted in time order, by constructing a representative subspace. The proper aging pattern for a previously unseen face image is determined by the projection in the subspace that can reconstruct the face image with minimum reconstruction error, while the position of the face image in that aging pattern will then indicate its age. In the experiments, AGES and its variants are compared with the limited existing age estimation methods (WAS and AAS) and some well-established classification methods (kNN, BP, C4.5, and SVM). Moreover, a comparison with human perception ability on age is conducted. It is interesting to note that the performance of AGES is not only significantly better than that of all the other algorithms, but also comparable to that of the human observers.
Improve Computer%2DAided Diagnosis With Machine Learning Techniques Using Undiagnosed Samples.
 In computer-aided diagnosis, machine learning techniques have been widely applied to learn hypothesis from diagnosed samples in order to assist the medical experts in making diagnosis. To learn a well-performed hypothesis, a large amount of diagnosed samples are required. Although the samples can be easily collected from routine medical examinations, it is usually impossible for the medical experts to make diagnosis for each of the collected samples. If hypothesis could be learned in presence of a large amount of undiagnosed samples, the heavy burden on the medical experts could be released. In this paper, a new semi-supervised learning algorithm named Co-Forest is proposed. It extends the co-training paradigm by using a well-known ensemble method named Random Forest, which enables Co-Forest to estimate the labeling confidence of undiagnosed samples and produce the final hypothesis easily. Experiments on benchmark data sets verify the effectiveness of the proposed algorithm. Case studies on three medical data sets and a successful application to microcalcification detection for breast cancer diagnosis show that undiagnosed samples are helpful in building computer-aided diagnosis systems, and Co-Forest is able to enhance the performance of the hypothesis learned on only a small amount of diagnosed samples by utilizing the available undiagnosed samples. 
Predicting Future Customers via Ensembling Gradually Expanded Trees.

Solving multi%2Dinstance problems with classifier ensemble based on constructive clustering.

ML%2DKNN A lazy learning approach to multi%2Dlabel learning.
Abstract: Multi-label learning originated from the investigation of text categorization problem, where each document may belong to several predefined topics simultaneously. In multi-label learning, the training set is composed of instances each associated with a set of labels, and the task is to predict the label sets of unseen instances through analyzing training instances with known label sets. In this paper, a multi-label lazy learning approach named Mlknn is presented, which is derived from the traditional k-Nearest Neighbor (kNN) algorithm. In detail, for each unseen instance, its k nearest neighbors in the training set are firstly identified. After that, based on statistical information gained from the label sets of these neighboring instances, i.e. the number of neighboring instances belonging to each possible class, maximum a posteriori (MAP) principle is utilized to determine the label set for the unseen instance. Experiments on three different real-world multi-label learning problems, i.e. Yeast gene functional analysis, natural scene classification and automatic web page categorization, show that Ml-knn achieves superior performance to some well-established multi-label learning algorithms.
A New Approach to Estimating the Expected First Hitting Time of Evolutionary Algorithms.
www.elsevier.com/locate/artint A new approach to estimating the expected first hitting time
Improve Web Search Using Image Snippets.

On Multi%2DClass Cost%2DSensitive Learning.
Cost-sensitive learning addresses the issue of classification in the presence of varying costs associated with different types of misclassification. In this paper, we present a method for solving multi-class cost-sensitive learning problems using any binary classification algorithm. This algorithm is derived using three key ideas: 1) iterative weighting; 2) expanding data space; and 3) gradient boosting with stochastic ensembles. We establish some theoretical guarantees concerning the performance of this method. In particular, we show that a certain variant possesses the boosting property, given a form of weak learning assumption on the component binary classifier. We also empirically evaluate the performance of the proposed method using benchmark data sets and verify that our method generally achieves better results than representative methods for cost-sensitive learning, in terms of predictive performance (cost minimization) and, in many cases, computational efficiency.
Learning from facial aging patterns for automatic age estimation.
potential applications in daily life. However, automatic age estimation technique is still underdeveloped. One of the main reasons is that the aging effects on human faces present several unique characteristics which make age estimation a challenging task that requires non-standard classification approaches. According to the speciality of the facial aging effects, this paper proposes the AGES (AGing pattErn Subspace) method for automatic age estimation. The basic idea is to model the aging pattern, which is defined as a sequence of personal aging face images, by learning a representative subspace. The proper aging pattern for an unseen face image is then determined by the projection in the subspace that can best reconstruct the face image, while the position of the face image in that aging pattern will indicate its age. The AGES method has shown encouraging performance in the comparative experiments either as an age estimator or as an age range estimator.
Generation of Comprehensible Hypotheses from Gene Expression Data.

Learning Non%2DMetric Partial Similarity Based on Maximal Margin Criterion.
The performance of many computer vision and machine learning algorithms critically depends on the quality of the similarity measure defined over the feature space. Previous works usually utilize metric distances which are often epistemologically different from the perceptual distance of human beings. In this paper, a novel non-metric partial similarity measure is introduced, which is born to automatically capture the prominent partial similarity between two images while ignoring the confusing unimportant dissimilarity. This measure is potentially useful in face recognition since it can help identify the inherent intra-personal similarity and thus reducing the influence caused by large variations such as expression and occlusions. Moreover, to make this method practical, this paper proposes an automatic and class-dependent similarity threshold setting mechanism based on the maximal margin criterion, and uses a Self-Organization Map-based embedding technique to alleviate the computational problem. Experimental results show the feasibility and effectiveness of the proposed method. 1
Exploratory Under%2DSampling for Class%2DImbalance Learning.
Under-sampling is a class-imbalance learning method which uses only a subset of major class examples and thus is very efficient. The main deficiency is that many major class examples are ignored. We propose two algorithms to overcome the deficiency. EasyEnsemble samples several subsets from the major class, trains a learner using each of them, and combines the outputs of those learners. BalanceCascade is similar toEasyEnsemble except that it removes correctly classified major class examples of trained learners from further consideration. Experiments show that both of the proposed algorithms have better AUC scores than many existing class-imbalance learning methods. Moreover, they have approximately the same training time as that of under-sampling, which trains significantly faster than other methods. 1
Distributional Features for Text Categorization.

Adaptive Kernel Principal Component Analysis with Unsupervised Learning of Kernels.
Choosing an appropriate kernel is one of the key problems in kernel-based methods. Most existing kernel selection methods require that the class labels of the training examples are known. In this paper, we propose an adaptive kernel selection method for kernel principal component analysis, which can effectively learn the kernels when the class labels of the training examples are not available. By iteratively optimizing a novel criterion, the proposed method can achieve nonlinear feature extraction and unsupervised kernel learning simultaneously. Moreover, a noniterative approximate algorithm is developed. The effectiveness of the proposed algorithms are validated on UCI datasets and the COIL-20 object recognition database. 1
Query%2DSensitive Similarity Measure for Content%2DBased Image Retrieval.

Exploratory Under%2DSampling for Class%2DImbalance Learning.
Under-sampling is a class-imbalance learning method which uses only a subset of major class examples and thus is very efficient. The main deficiency is that many major class examples are ignored. We propose two algorithms to overcome the deficiency. EasyEnsemble samples several subsets from the major class, trains a learner using each of them, and combines the outputs of those learners. BalanceCascade is similar toEasyEnsemble except that it removes correctly classified major class examples of trained learners from further consideration. Experiments show that both of the proposed algorithms have better AUC scores than many existing class-imbalance learning methods. Moreover, they have approximately the same training time as that of under-sampling, which trains significantly faster than other methods. 1
The Influence of Class Imbalance on Cost%2DSensitive Learning An Empirical Study.

Multi%2DInstance Multi%2DLabel Learning with Application to Scene Classification.
In this paper, we formalize multi-instance multi-label learning, where each training example is associated with not only multiple instances but also multiple class labels. Such a problem can occur in many real-world tasks, e.g. an image usually contains multiple patches each of which can be described by a feature vector, and the image can belong to multiple categories since its semantics can be recognized in different ways. We analyze the relationship between multi-instance multi-label learning and the learning frameworks of traditional supervised learning, multiinstance learning and multi-label learning. Then, we propose the MIMLBOOST and MIMLSVM algorithms which achieve good performance in an application to scene classification. 1
Neighbor Line%2DBased Locally Linear Embedding.
Abstract. Locally linear embedding (Lle) is a powerful approach for mapping high-dimensional data nonlinearly to a lower-dimensional space. However, when the training examples are not densely sampled, Lle often returns invalid results. In this paper, the Nl 3 e (Neighbor Line-based Lle) approach is proposed, which generates some virtual examples with the help of neighbor line such that the Lle learning can be executed on an enriched training set. Experiments show that Nl 3 e outperforms Lle in visualization. 1
Non%2Dnegative Matrix Factorization on Kernels.
Abstract. In this paper, we extend the original non-negative matrix factorization (NMF) to kernel NMF (KNMF). The advantages of KNMF over NMF are: 1) it could extract more useful features hidden in the original data through some kernel-induced nonlinear mappings; 2) it can deal with data where only relationships (similarities or dissimilarities) between objects are known; 3) it can process data with negative values by using some specific kernel functions (e.g. Gaussian). Thus, KNMF is more general than NMF. To further improve the performance of KNMF, we also propose the SpKNMF, which performs KNMF on sub-patterns of the original data. The effectiveness of the proposed algorithms is validated by extensive experiments on UCI datasets and the FERET face database. 1
Learning with Unlabeled Data and Its Application to Image Retrieval.

Uncontrolled Face Recognition by Individual Stable Neural Network.
Abstract. There usually exist diverse variations in face images taken under uncontrolled conditions. Most previous work on face recognition focuses on particular variations and usually assume the absence of others. Such work is called controlled face recognition. Instead of the ‘divide and conquer ’ strategy adopted by controlled face recognition, this paper presents one of the first attempts directly aiming at uncontrolled face recognition. The solution is based on Individual Stable Neural Network (ISNN) proposed in this paper. ISNN can map a face image into the so-called Individual Stable Space (ISS), the feature space that only expresses personal characteristics, which is the only useful information for recognition. There are no restrictions for the face images fed into ISNN. Moreover, unlike many other robust face recognition methods, ISNN does not require any extra information (such as view angle) other than the personal identities during training. These advantages of ISNN make it a very practical approach for uncontrolled face recognition. In the experiments, ISNN is tested on two large face databases with vast variations and achieves the best performance compared with several popular face recognition techniques. 1
Recognizing Face or Object from a Single Image Linear vs. Kernel Methods on 2D Patterns.
Abstract. We consider the problem of recognizing face or object when only single training image per class is available, which is typically encountered in law enforcement, passport or identification card verification, etc. In such cases, many discriminant subspace methods such as Linear Discriminant Analysis (LDA) fail because of the non-existence of intra-class variation. In this paper, we propose a novel framework called 2-Dimensional Kernel PCA (2D-KPCA) for face or object recognition from a single image. In contrast to conventional KPCA, 2D-KPCA is based on 2D image matrices and hence can effectively utilize the intrinsic spatial structure information of the images. On the other hand, in contrast to 2D-PCA, 2D-KPCA is capable of capturing part of the higher-order statistics information. Moreover, this paper reveals that the current 2D-PCA algorithm and its many variants consider only the row information or column information, which has not fully exploited the information contained in the image matrices. So, besides proposing the unilateral 2D-KPCA, this paper also proposes the bilateral 2D-KPCA which could exploit more information concealed in the image matrices Furthermore, some approximation techniques are developed for improving the computational efficiency. Experimental results on the FERET face database and the COIL-20 object database show that: 1) the performance of KPCA is not necessarily better than that of PCA; 2) 2D-KPCA almost always outperforms 2D-PCA significantly; 3) the kernel methods are more appropriate on 2D pattern than on 1D patterns. 1
Enhancing relevance feedback in image retrieval using unlabeled data.
Relevance feedback is an effective scheme bridging the gap between high-level semantics and lowlevel features in content-based image retrieval (Cbir). In contrast to previous methods which rely on labeled images provided by the user, this paper attempts to enhance the performance of relevance feedback by exploiting unlabeled images existing in the database. Concretely, this paper integrates the merits of semi-supervised learning and active learning into the relevance feedback process. In detail, in each round of relevance feedback, two simple learners are trained from the labeled data, i.e. images from user query and user feedback. Each learner then labels some unlabeled images in the database for the other learner. After re-training with the additional labeled data, the learners classify the images in the database again and then their classifications are merged. Images judged to be positive with high confidence are returned as the retrieval result, while those judged with low confidence are put into the pool which is used in the next round of relevance feedback. Experiments show that using semi-supervised learning and active learning simultaneously in Cbir is beneficial, and the proposed method achieves better performance than some existing methods.
Training Cost%2DSensitive Neural Networks with Methods Addressing the Class Imbalance Problem.

Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization.
Abstract — In multi-label learning, each instance in the training set is associated with a set of labels, and the task is to output a label set whose size is unknown a priori for each unseen instance. In this paper, this problem is addressed in the way that a neural network algorithm named BP-MLL, i.e. Backpropagation for Multi-Label Learning, is proposed. It is derived from the popular Backpropagation algorithm through employing a novel error function capturing the characteristics of multi-label learning, i.e. the labels belonging to an instance should be ranked higher than those not belonging to that instance. Applications to two realworld multi-label learning problems, i.e. functional genomics and text categorization, show that the performance of BP-MLL is superior to those of some well-established multi-label learning algorithms.
Image Region Selection and Ensemble for Face Recognition.
Abstract In this paper, a novel framework for face recognition, namely Selective Ensemble of Image Regions (SEIR), is proposed. In this framework, all possible regions in the face image are regarded as a certain kind of features. There are two main steps in SEIR: the first step is to automatically select several regions from all possible candidates; the second step is to construct classifier ensemble from the selected regions. A realization of SEIR based on multiple eigenspaces, namely SEME, is also proposed in this paper. SEME is analyzed and compared with eigenface, PCA + LDA, eigenfeature, and eigenface + eigenfeature through experiments. The experimental results show that SEME achieves the best performance.
Multi%2DInstance Learning from Supervised View.
Abstract In multi-instance learning, the training set comprises labeled bags that are composed of unlabeled instances, and the task is to predict the labels of unseen bags. This paper studies multi-instance learning from the view of supervised learning. First, by analyzing some representative learning algorithms, this paper shows that multi-instance learners can be derived from supervised learners by shifting their focuses from the discrimination on the instances to the discrimination on the bags. Second, considering that ensemble learning paradigms can effectively enhance supervised learners, this paper proposes to build multi-instance ensembles to solve multi-instance problems. Experiments on a real-world benchmark test show that ensemble learning paradigms can significantly enhance multi-instance learners.
Clusterer ensemble.
Cluster ensembles are collections of individual solutions to a given clustering problem which are useful or necessary to consider in a wide range of applications. The R package clue provides an extensible computational environment for creating and analyzing cluster ensembles, with basic data structures for representing partitions and hierarchies, and facilities for computing on these, including methods for measuring proximity and obtaining consensus and “secondary ” clusterings. 1
Adapting RBF Neural Networks to Multi%2DInstance Learning.
Abstract. In multi-instance learning, the training examples are bags composed of instances without labels, and the task is to predict the labels of unseen bags through analyzing the training bags with known labels. A bag is positive if it contains at least one positive instance, while it is negative if it contains no positive instance. In this paper, a neural network based multi-instance learning algorithm named RBF-MIP is presented, which is derived from the popular radial basis function (RBF) methods. Briefly, the first layer of an RBF-MIP neural network is composed of clusters of bags formed by merging training bags agglomeratively, where Hausdorff metric is utilized to measure distances between bags and between clusters. Weights of second layer of the RBF-MIP neural network are optimized by minimizing a sum-of-squares error function and worked out through singular value decomposition (SVD). Experiments on real-world multi-instance benchmark data, artificial multi-instance benchmark data and natural scene image database retrieval are carried out. The experimental results show that RBF-MIP is among the several best learning algorithms on multi-instance problems. Key words. Machine learning, Multi-instance learning, Radial basis function, Neu-ral networks, Hausdorff distance, Singular value decomposition, Principle component analysis, Content-based image retrieval 1.
Guest Editors%27 introduction to the special issue machine learning approaches to multimedia information retrieval.

Adapting RBF Neural Networks to Multi%2DInstance Learning.
Abstract. In multi-instance learning, the training examples are bags composed of instances without labels, and the task is to predict the labels of unseen bags through analyzing the training bags with known labels. A bag is positive if it contains at least one positive instance, while it is negative if it contains no positive instance. In this paper, a neural network based multi-instance learning algorithm named RBF-MIP is presented, which is derived from the popular radial basis function (RBF) methods. Briefly, the first layer of an RBF-MIP neural network is composed of clusters of bags formed by merging training bags agglomeratively, where Hausdorff metric is utilized to measure distances between bags and between clusters. Weights of second layer of the RBF-MIP neural network are optimized by minimizing a sum-of-squares error function and worked out through singular value decomposition (SVD). Experiments on real-world multi-instance benchmark data, artificial multi-instance benchmark data and natural scene image database retrieval are carried out. The experimental results show that RBF-MIP is among the several best learning algorithms on multi-instance problems. Key words. Machine learning, Multi-instance learning, Radial basis function, Neu-ral networks, Hausdorff distance, Singular value decomposition, Principle component analysis, Content-based image retrieval 1.


Diagonal principal component analysis for face recognition.
In this paper, a novel subspace method called diagonal principal component analysis (DiaPCA) is proposed for face recognition. In contrast to standard PCA, DiaPCA directly seeks the optimal projective vectors from diagonal face images without image-to-vector transformation. While in contrast to 2DPCA, DiaPCA reserves the correlations between variations of rows and those of columns of images. Experiments show that DiaPCA is much more accurate than both PCA and 2DPCA. Furthermore, it is shown that the accuracy can be further improved by combining DiaPCA with 2DPCA.
Face recognition from a single image per person A survey.
Abstract One of the main challenges faced by the current face recognition techniques lies in the difficulties of collecting samples. Fewer samples per person mean less laborious effort for collecting them, lower costs for storing and processing them. Unfortunately, many reported face recognition techniques rely heavily on the size and representative of training set, and most of them will suffer serious performance drop or even fail to work if only one training sample per person is available to the systems. This situation is called “one sample per person ” problem: given a stored database of faces, the goal is to identify a person from the database later in time in any different and unpredictable poses, lighting, etc from just one image. Such a task is very challenging for most current algorithms due to the extremely limited representative of training sample. Numerous techniques have been developed to attack this problem, and the purpose of this paper is to categorize and evaluate these algorithms. The prominent algorithms are described and critically analyzed. Relevant issues such as data collection, the influence of the small sample size, and system evaluation are discussed, and several promising directions for future research are also proposed in this paper.
Two%2DDimensional Non%2Dnegative Matrix Factorization for Face Representation and Recognition.
Abstract. Non-negative matrix factorization (NMF) is a recently developed method for finding parts-based representation of non-negative data such as face images. Although it has successfully been applied in several applications, directly using NMF for face recognition often leads to low performance. Moreover, when performing on large databases, NMF needs considerable computational costs. In this paper, we propose a novel NMF method, namely 2DNMF, which stands for 2-D non-negative matrix factorization. The main difference between NMF and 2DNMF is that the former first align images into 1D vectors and then represents them with a set of 1D bases, while the latter regards images as 2D matrices and represents them with a set of 2D bases. Experimental results on several face databases show that 2DNMF has better image reconstruction quality than NMF under the same compression ratio. Also the running time of 2DNMF is less, and the recognition accuracy higher than that of NMF. 1
Locating Regions of Interest in CBIR with Multi%2Dinstance Learning Techniques.
Abstract. In content-based image retrieval (CBIR), the user usually poses several labelled images and then the system attempts to retrieve all the images relevant to the target concept defined by these labelled images. It may be helpful if the system can return relevant images where the regions of interest (ROI) are explicitly located. In this paper, this task is accomplished with the help of multi-instance learning techniques. In detail, this paper proposes the CkNN-ROI algorithm, which regards each image as a bag comprising many instances and picks from positive bag the instance that has great chance to meet the target concept to help locate ROI. Experiments show that the proposed algorithm can efficiently locate ROI in CBIR process. 1
A k%2Dnearest neighbor based algorithm for multi%2Dlabel classification.

An Intelligent Medical Image Understanding Method Using Two%2DTier Neural Network Ensembles.

Semi%2DSupervised Regression with Co%2DTraining.
The traditional setting of supervised learning requires a large amount of labeled training examples in order to achieve good generalization. However, in many practical applications, unlabeled training examples are readily available but labeled ones are fairly expensive to obtain. Therefore, semi-supervised learning has attracted much attention. Previous research on semi-supervised learning mainly focuses on semi-supervised classification. Although regression is almost as important as classification, semi-supervised regression is largely understudied. In particular, although co-training is a main paradigm in semi-supervised learning, few works has been devoted to co-training style semi-supervised regression algorithms. In this paper, a co-training style semi-supervised regression algorithm, i.e. COREG, is proposed. This algorithm uses two regressors each labels the unlabeled data for the other regressor, where the confidence in labeling an unlabeled example is estimated through the amount of reduction in mean square error over the labeled neighborhood of that example. Analysis and experiments show that COREG can effectively exploit unlabeled data to improve regression estimates.
Spiculated Lesion Detection in Digital Mammogram Based on Artificial Neural Network Ensemble.

Feature Selection for High Dimensional Face Image Using Self%2Dorganizing Maps.
Abstract: While feature selection is very difficult for high dimensional, unstructured data such as face image, it may be much easier to do if the data can be faithfully transformed into lower dimensional space. In this paper, a new method is proposed to transform the high dimensional face images into low-dimensional SOM topological space, and then identify important local features of face images for face recognition automatically using simple statistics computed from the class distribution of the face image data. The effectiveness of the proposed method are demonstrated by the experiments on AR face databases, which reveal that up to 80 % local features can be pruned with only slightly loss of the classification accuracy. 1.
SETRED Self%2Dtraining with Editing.
Self-training is a semi-supervised learning algorithm in which  a learner keeps on labeling unlabeled examples and retraining itself on  an enlarged labeled training set. Since the self-training process may erroneously  label some unlabeled examples, sometimes the learned hypothesis  does not perform well. In this paper, a new algorithm named Setred  is proposed, which utilizes a specific data editing method to identify and  remove the mislabeled examples from the self-labeled data. In detail, in  each iteration of the self-training process, the local cut edge weight statistic  is used to help estimate whether a newly labeled example is reliable  or not, and only the reliable self-labeled examples are used to enlarge  the labeled training set. Experiments show that the introduction of data  editing is beneficial, and the learned hypotheses of Setred outperform  those learned by the standard self-training algorithm.
Dependency Bagging.
The modified quark-meson coupling model, which features a density dependent bag constant and bag radius in nuclear matter, is checked against the EMC effect within the framework of dynamical rescaling. Our emphasis is on the change in the average bag radius in nuclei, as evaluated in a local density approximation, and its implication for the rescaling parameter. We find that when the bag constant in nuclear matter is significantly reduced from its free-space value, the resulting rescaling parameter is in good agreement with that required to explain the observed depletion of the structure functions in the medium Bjorken x region. Such a large reduction of the bag constant also implies large and canceling Lorentz scalar and vector potentials for the nucleon in nuclear matter which are comparable to those suggested by the relativistic nuclear phenomenology and finite-density QCD sum rules. PACS numbers: 24.85+p; 25.30.Mr; 13.60.Hb; 12.39.Ba Typeset using REVT E X (submitted to Physical Re...
Multi%2DInstance Learning Based Web Mining.
In multi-instance learning, the training set comprises labeled bags that are composed of unlabeled instances, and the task is to predict the labels of unseen bags. In this paper, a web mining problem, i.e. web index recommendation, is investigated from a multi-instance view. In detail, each web index page is regarded as a bag, while each of its linked pages is regarded as an instance. A user favoring an index page means that he or she is interested in at least one page linked by the index. Based on the browsing history of the user, recommendation could be provided for unseen index pages. An algorithm named Fretcit-kNN, which employs the Minimal Hausdorff distance between frequent term sets and utilizes both the references and citers of an unseen bag in determining its label, is proposed to solve the problem. Experiments show that in average the recommendation accuracy of Fretcit-kNN is 81.0% with 71.7% recall and 70.9% precision, which is significantly better than the best algorithm that does not consider the specific characteristics of multi-instance learning, whose performance is 76.3% accuracy with 63.4% recall and 66.1% precision.
A new face recognition method based on SVD perturbation for single example image per person.
At present, there are many methods for frontal view face recognition. However, few of them can work well when only one example image per class is available. In this paper, we present a new method based on SVD perturbation to deal with the &#039;one example image &#039; problem and two generalized eigenface algorithms are proposed. In the first algorithm, the original image is linearly combined with its derived image gotten by perturbing the image matrix&#039;s singular values, and then principal component analysis (PCA) is performed on the joined images. In the second algorithm, the derived images are regarded as independent images that could augment training image set, and then PCA is performed on all the training images available, including the original ones and the derived ones. The proposed algorithms are compared with both the standard eigenface algorithm and the (PC) 2 A algorithm which is proposed for addressing the ‘one example image ’ problem, on the well-known FERET database with three different image resolutions. Experimental results show that the generalized eigenface algorithms are more accurate and use far fewer eigenfaces than both the standard eigenface algorithm and the (PC) 2 A algorithm.
Tri%2DTraining Exploiting Unlabeled Data Using Three Classifiers.

Ensembling local learners ThroughMultimodal perturbation.

Supervised nonlinear dimensionality reduction for visualization and classification.
Abstract—When performing visualization and classification, people often confront the problem of dimensionality reduction. Isomap is one of the most promising nonlinear dimensionality reduction techniques. However, when Isomap is applied to real-world data, it shows some limitations, such as being sensitive to the noise. In this paper, an improved version of Isomap, namely S-Isomap, is proposed. S-Isomap utilizes class information to guide the procedure of nonlinear dimensionality reduction. Such a kind of procedure is called supervised nonlinear dimensionality reduction. In S-Isomap, the neighborhood graph of the input data is constructed according to a certain kind of dissimilarity between data points, which is specially designed to integrate the class information. The dissimilarity has several good properties which help to discover the true neighborhood of the data, and thus makes S-Isomap a robust technique for both visualization and classification, especially for real-world problems. In the visualization experiments, S-Isomap is compared with Isomap, LLE and WeightedIso. The results show that S-Isomap performs the best. In the classification experiments, S-Isomap is used as a preprocess of classification and compared with Isomap, WeightedIso, as well as some other well-established classification methods including K nearest neighbor classifier, BP neural network, J4.8 decision tree and SVM. The results reveal that S-Isomap excels Isomap and WeightedIso in classification and is highly competitive with those well-known classification methods. Index Terms—Supervised learning, Dimensionality reduction, Manifold learning, Visualization, Classification
Adapt Bagging to Nearest Neighbor Classifiers.
It is well-known that in order to build a strong ensemble, the component learners should be with high diversity as well as high accuracy. If perturbing the training set can cause significant changes in the component learners constructed, then Bagging can effectively improve accuracy. However, for stable learners such as nearest neighbor classifiers, perturbing the training set can hardly produce diverse component learners, therefore Bagging does not work well. This paper adapts Bagging to nearest neighbor classifiers through injecting randomness to distance metrics. In detail, in constructing the component learners, both the training set and the distance metric employed for identifying the neighbors are perturbed. A large empirical study reported in this paper shows that the proposed BagInRand algorithm can effectively improve the accuracy of nearest neighbor classifiers.
A unified SWSI%2DKAMs framework and performance evaluation on face recognition.
Abstract: Kernel method is an effective and popular trick in machine learning. In this paper, by introducing it into conventional auto-associative memory models (AMs), we construct a unified framework of kernel auto-associative memory models (KAMs),which makes the existing exponential and polynomial AMs become its special cases. Further, in order to reduce KAM’s connect complexity, inspired by “small-world network ” recently described by Watts and Strogatz, we propose another unified framework of small-world structure (SWS) inspired kernel auto-associative memory models (SWSI-KAMs), which, in principle, makes KAMs implemented easier in structure. Simulation results on FERET face database show that, the SWSI-KAMs adopting such kernels as Exponential and Hyperbolic tangent kernels have advantages of configuration simplicity while their recognition performance is almost as well as, even better than, corresponding KAMs with full connectivity. In the end, the SWSI-KAM adopting Exponential kernel with different connectivities was emphatically investigated for robustness based on those face images which are added random noises and/or partially occluded in a mosaic way, and the experiments demonstrate that the SWSI-KAM with Exponential kernel is more robust in all cases of network connectivity of 20%, 40 % and 60 % than
%282D%292PCA Two%2Ddirectional two%2Ddimensional PCA for efficient face representation and recognition.

Meta%2Dgame Equilibrium for Multi%2Dagent Reinforcement Learning.
This paper proposes a multi-agent Q-learning algorithm called  meta-game-Q learning that is developed from the meta-game equilibrium  concept. Di#erent from Nash equilibrium, meta-game equilibrium can  achieve the optimal joint action game through deliberating its preference  and predicting others&#039; policies in the general-sum game. A distributed negotiation  algorithm is used to solve the meta-game equilibrium problem  instead of using centralized linear programming algorithms. We use the  repeated prisoner&#039;s dilemma example to empirically demonstrate that  the algorithm converges to meta-game equilibrium.
Exploiting Unlabeled Data in Content%2DBased Image Retrieval.
In this paper, the SSAIR (Semi-Supervised Active Image Retrieval) approach, which attempts to exploit unlabeled data to improve the performance of content-based image retrieval (CBIR), is proposed. This approach combines the merits of semi-supervised learning and active learning. In detail, in each round of relevance feedback, two simple learners are trained from the labeled data, i.e. images from user query and user feedback. Each learner then classifies the unlabeled images in the database and passes the most relevant/irrelevant images to the other learner. After re-training with the additional labeled data, the learners classify the images in the database again and then their classifications are merged. Images judged to be relevant with high confidence are returned as the retrieval result, while these judged with low confidence are put into the pool which is used in the next round of relevance feedback. Experiments show that semi-supervised learning and active learning mechanisms are both beneficial to CBIR.
Exploiting Unlabeled Data in Content%2DBased Image Retrieval.
In this paper, the SSAIR (Semi-Supervised Active Image Retrieval) approach, which attempts to exploit unlabeled data to improve the performance of content-based image retrieval (CBIR), is proposed. This approach combines the merits of semi-supervised learning and active learning. In detail, in each round of relevance feedback, two simple learners are trained from the labeled data, i.e. images from user query and user feedback. Each learner then classifies the unlabeled images in the database and passes the most relevant/irrelevant images to the other learner. After re-training with the additional labeled data, the learners classify the images in the database again and then their classifications are merged. Images judged to be relevant with high confidence are returned as the retrieval result, while these judged with low confidence are put into the pool which is used in the next round of relevance feedback. Experiments show that semi-supervised learning and active learning mechanisms are both beneficial to CBIR.
Fuzzy%2DKernel Learning Vector Quantization.

Editing Training Data for kNN Classifiers with Neural Network Ensemble.
Since kNN classifiers are sensitive to outliers and noise contained in the training data set, many approaches have been proposed to edit the training data so that the performance of the classifiers can be improved. In this paper, through detaching the two schemes adopted by the Depuration algorithm, two new editing approaches are derived. Moreover, this paper proposes to use neural network ensemble to edit the training data for kNN classifiers. Experiments show that such an approach is better than the approaches derived from Depuration, while these approaches are better than or comparable to Depuration.
Progressive Principal Component Analysis.

Robust Face Recognition from a Single Training Image per Person with Kernel%2DBased SOM%2DFace.
Abstract. In this paper, a kernel-based SOM-face method is proposed to recognize expression variant faces under the situation of only one training image per person. Based on the localization of the face, an unsupervised kernel-SOM learning procedure is carried out to capture the common local features and the non-Euclidean structure of the image data, so that a compact and robust representation of the face can be obtained. Experiments on the FERET face database show that the Kernel-based SOM-face method can obtain higher recognition performance than the regular SOM-face method. 1
Ensembles of Multi%2DInstance Neural Networks.

On the Size of Training Set and the Benefit from Ensemble.
In this paper, the impact of the size of the training set on the benefit from ensemble, i.e. the gains obtained by employing ensemble learning paradigms, is empirically studied. Experiments on Bagged/ Boosted J4.8 decision trees with/without pruning show that enlarging the training set tends to improve the benefit from Boosting but does not significantly impact the benefit from Bagging. This phenomenon is then explained from the view of bias-variance reduction. Moreover, it is shown that even for Boosting, the benefit does not always increase consistently along with the increase of the training set size since single learners sometimes may learn relatively more from additional training data that are randomly provided than ensembles do. Furthermore, it is observed that the benefit from ensemble of unpruned decision trees is usually bigger than that from ensemble of pruned decision trees. This phenomenon is then explained from the view of error-ambiguity balance.
Ensembling MML Causal Discovery.
This paper presents an ensemble MML approach for the discovery of causal models. The component learners are formed based on the MML causal induction methods. Six different ensemble causal induction algorithms are proposed. Our experiential results reveal that (1) the ensemble MML causal induction approach has achieved an improved result compared with any single learner in terms of learning accuracy and correctness; (2) Among all the ensemble causal induction algorithms examined, the weighted voting without seeding algorithm outperforms all the rest; (3) It seems that the ensembled CI algorithms could alleviate the local minimum problem. The only drawback of this method is that the time complexity is increased by \delta times, where \delta is the ensemble size.
On the Size of Training Set and the Benefit from Ensemble.
In this paper, the impact of the size of the training set on the benefit from ensemble, i.e. the gains obtained by employing ensemble learning paradigms, is empirically studied. Experiments on Bagged/ Boosted J4.8 decision trees with/without pruning show that enlarging the training set tends to improve the benefit from Boosting but does not significantly impact the benefit from Bagging. This phenomenon is then explained from the view of bias-variance reduction. Moreover, it is shown that even for Boosting, the benefit does not always increase consistently along with the increase of the training set size since single learners sometimes may learn relatively more from additional training data that are randomly provided than ensembles do. Furthermore, it is observed that the benefit from ensemble of unpruned decision trees is usually bigger than that from ensemble of pruned decision trees. This phenomenon is then explained from the view of error-ambiguity balance.
Unified Locally Linear Embedding and Linear Discriminant Analysis Algorithm %28ULLELDA%29 for Face Recognition.
Manifold learning approaches such as locally linear embedding algorithm  (LLE) and isometric mapping (Isomap) algorithm are aimed to discover  the intrinsical low dimensional variables from high-dimensional nonlinear data.
Analyzing Magnification Factors and Principal Spread Directions in Manifold Learning.
Great amount of data under varying intrinsic features is thought of as high dimensional nonlinear  manifold in the observation space. How to analyze the mapping relationship between the high dimensional  manifold and the corresponding intrinsic low dimensional one quantitatively is important to machine  learning and cognitive science. In this paper, we propose SVD (singular value decomposition) based magnification  factors and spread direction for quantitative analyzing the relationship. The result of conducting  experiments on several databases show the advantages of this proposed SVD-based approach in manifold  learning.
Artificial intelligence in medicine in China.

NeC4.5 Neural Ensemble Based C4.5.
Decision tree is with good comprehensibility while neural network ensemble is with strong generalization ability. In this paper, these merits are integrated into a novel decision tree algorithm NeC4.5. This algorithm trains a neural network ensemble at first. Then, the trained ensemble is employed to generate a new training set through replacing the desired class labels of the original training examples with those output from the trained ensemble. Some extra training examples are also generated from the trained ensemble and added to the new training set. Finally, a C4.5 decision tree is grown from the new training set. Since its learning results are decision trees, the comprehensibility of NeC4.5 is better than that of neural network ensemble. Moreover, experiments show that the generalization ability of NeC4.5 decision trees can be better than that of C4.5 decision trees.
Rule Extraction Using Neural Networks or for Neural Networks%3F
In the research of rule extraction from neural networks, fidelity describes how well the rules mimic the behavior of a neural network while accuracy describes how well the rules can be generalized. This paper identifies the fidelity-accuracy dilemma. It argues to distinguish rule extraction using neural networks and rule extraction for neural networks according to their differing goals, where fidelity and accuracy should be excluded from the rule quality evaluation framework, respectively.
Improve Multi%2DInstance Neural Networks through Feature Selection.
Abstract. Multi-instance learning is regarded as a new learning framework where the training examples are bags composed of instances without labels, and the task is to predict the labels of unseen bags through analyzing the training bags with known labels. Recently, a multi-instance neural network named BP-MIP was proposed. In this paper, BP-MIP is improved through adopting two different feature selection techniques, i.e. feature scaling with Diverse Density and feature reduction with principal component analysis. In detail, before feature vectors are fed to a BP-MIP neural network, they are scaled by the feature weights found by running Diverse Density on the training data, or projected by a linear transformation matrix formed by principal component analysis. Experiments show that these feature selection mechanisms can significantly improve the performance of BP-MIP. Key words. Multi-instance learning, machine learning, neural networks, feature selection, backpropagation
SOM Ensemble%2DBased Image Segmentation.
Abstract. Image segmentation plays an important role in image analysis and image understanding. In this paper, an image segmentation method based on ensemble of SOM neural networks is proposed, which clusters the pixels in an image according to color and spatial features with many SOM neural networks, and then combines the clustering results to give the final segmentation. Experimental results show that the proposed method performs better than some existing clustering-based image segmentation methods. Key words. Image segmentation, SOM, neural networks, neural network ensemble, ensemble learning, unsupervised learning
Projection functions for eye detection.

Making FLDA applicable to face recognition with one sample per person.

Enhanced %28PC%292 A for face recognition with one training image per person
Abstract: Recently, a method called (PC) 2 A was proposed to deal with face recognition with one training image per person. As an extension of the standard eigenface technique, (PC) 2 A combines linearly each original face image with its corresponding first-order projection into a new face and then performs principal component analysis (PCA) on a set of the newly combined (training) images. It was reported that (PC) 2 A could achieve higher accuracy than the eigenface technique through using 10%-15 % fewer eigenfaces. In this paper, we generalize and further enhance (PC) 2 A along two directions. In the first direction, we combine the original image with its second-order projections as well as its first-order projection in order to acquire more information from the original face, and then similarly apply PCA to such a set of the combined images. In the second direction, instead of combining them, we still regard the projections of each original image as single derived images to augment training image set, and then perform PCA on all the training images available, including the original ones and the derived ones. Experiments on the well-known FERET database show that the enhanced versions of (PC) 2 A are about 1.6 % to 3.5 % more accurate and
Ensembles of Multi%2Dinstance Learners.
Abstract. In multi-instance learning, the training set comprises labeled bags that are composed of unlabeled instances, and the task is to predict the labels of unseen bags. Through analyzing two famous multi-instance learning algorithms, this paper shows that many supervised learning algorithms can be adapted to multi-instance learning, as long as their focuses are shifted from the discrimination on the instances to the discrimination on the bags. Moreover, considering that ensemble learning paradigms can effectively enhance supervised learners, this paper proposes to build ensembles of multi-instance learners to solve multi-instance problems. Experiments on a real-world benchmark test show that ensemble learning paradigms can significantly enhance multi-instance learners, and the result achieved by EM-DD ensemble exceeds the best result on the benchmark test reported in literature. 1
A Novel Bag Generator for Image Database Retrieval With Multi%2DInstance Learning Techniques.
In multi-instance learning, the training examples are bags composed of instances without labels and the task is to predict the labels of unseen bags through analyzing the training bags with known labels. In content-based image retrieval (CBIR), the query is ambiguous because it is not feasible to ask the user precisely specify what he or she wants. Such kind of ambiguity can be gracefully dealt with by multi-instance learning techniques, and previous research shows that bag generators can significantly influence the performance of a CBIR system. In this paper, a novel bag generator named ImageBag is presented, where the pixels of each image are first clustered based on their color and spatial features and then the clustered blocks are merged and converted into a specific number of instances. Experiments show that ImageBag achieves comparable results to some existing bag generators but is more efficient in retrieving images from database.
Selective Ensemble of Decision Trees.
An ensemble is generated by training multiple component learners for a same task and then combining their predictions. In most ensemble algorithms, all the trained component learners are employed in constituting an ensemble. But recently, it has been shown that when the learners are neural networks, it may be better to ensemble some instead of all of the learners. In this paper, this claim is generalized to situations where the component learners are decision trees. Experiments show that ensembles generated by a selective ensemble algorithm, which selects some of the trained C4.5 decision trees to make up an ensemble, may be not only smaller in the size but also stronger in the generalization than ensembles generated by non-selective algorithms.
SOM Based Image Segmentation.
Image segmentation plays an important role in image retrieval system. In this paper, a method for segmenting images based on SOM neural network is proposed. At first, the pixels are clustered based on their color and spatial features, where the clustering process is accomplished with a SOM network. Then, the clustered blocks are merged to a specific number of regions. Experiments show that these regions could be regarded as segmentation results reserving some semantic means. This approach thus provides a feasible new solution for image segmentation which may be helpful in image retrieval.
Extracting symbolic rules from trained neural network ensembles.
Neural network ensemble can significantly improve the generalization ability of neural network based systems. However, its comprehensibility is even worse than that of a single neural network because it comprises a collection of individual neural networks. In this paper, an approach named REFNE is proposed to improve the comprehensibility of trained neural network ensembles that perform classification tasks. REFNE utilizes the trained ensembles to generate instances and then extracts symbolic rules from those instances. It gracefully breaks the ties made by individual neural networks in prediction. It also employs specific discretization scheme, rule form, and fidelity evaluation mechanism. Experiments show that with different configurations, REFNE can extract rules with good fidelity that well explain the function of trained neural network ensembles, or rules with strong generalization ability that are even better than the trained neural network ensembles in prediction.
Three perspectives of data mining.
This paper reviews three recent books on data mining written from three different perspectives, i.e. databases, machine learning, and statistics. Although the exploration in this paper is suggestive instead of conclusive, it reveals that besides some common properties, different perspectives lay strong emphases on different aspects of data mining. The emphasis of the database perspective is on efficiency because this perspective strongly concerns the whole discovery process and huge data volume. The emphasis of the machine learning perspective is on effectiveness because this perspective is heavily attracted by substantive heuristics working well in data analysis although they may not always be useful. As for the statistics perspective, its emphasis is on validity because this perspective cares much for mathematical soundness behind mining methods.
Medical diagnosis with C4.5 rule preceded by artificial neural network ensemble.
Comprehensibility is very important for a machine learning technique to be used in computer-aided medical diagnosis. Since an artificial neural network ensemble is composed of multiple artificial neural networks, its comprehensibility is worse than that of a single artificial neural network. In this paper, C4.5 Rule-PANE which combines artificial neural network ensemble with rule induction by regarding the former as a pre-process of the latter, is proposed. At first, an artificial neural network ensemble is trained. Then, a new training data set is generated by feeding the feature vectors of the original training instances to the trained ensemble and replacing the expected class labels of the original training instances with the class labels output from the ensemble. Additional training data may also be appended by randomly generating feature vectors and combining them with their corresponding class labels output from the ensemble. Finally, a specific rule induction approach, i.e. C4.5 Rule, is used to learn rules from the new training data set. Case studies on diabetes, hepatitis, and breast cancer show that C4.5 Rule-PANE could generate rules with strong generalization ability, which profits from artificial neural network ensemble, and strong comprehensibility, which profits from rule induction.

Evolving Fault%2DTolerant Neural Networks.
In this paper, genetic algorithm is used to help improve the tolerance of feedforward neural networks against open fault. The proposed method does not explicitly add any redundancy to the network, nor does it modify the training algorithm. Experiments show that it may profit the fault tolerance as well as the generalization ability of neural networks.
Efficient face candidates selector for face detection.
In this paper an efficient face candidates selector is proposed for face detection tasks in still gray level images. The proposed method acts as a selective attentional mechanism. Eye-analogue segments at a given scale are discovered by finding regions which are roughly as large as real eyes and are darker than their neighborhoods. Then a pair of eye-analogue segments are hypothesized to be eyes in a face and combined into a face candidate if their placement is consistent with the anthropological characteristic of human eyes. The proposed method is robust in that it can deal with illumination changes and moderate rotations. A subset of the FERET data set and the BioID face database are used to evaluate the proposed method. The proposed face candidates selector is successful in 98.75% and 98.6% cases respectively.
The Application of Visualization and Neural Network Techniques in a Power Transformer Condition Monitoring System.
In this paper, visualization and neural network techniques are applied together to a power transformer condition monitoring system. Through visualizing the data from the chromatogram of oil-dissolved gases by 2-D and/or 3-D graphs, the potential failures of the power transformers become easy to be identified. Through employing some specific neural network techniques, the data from the chromatogram of oil-dissolved gases as well as those from the electrical inspections can be effectively analyzed. Experiments show that the described system works quite well in condition monitoring of power transformers.

Lung cancer cell identification based on artificial neural network ensembles.
An artificial neural network ensemble is a learning paradigm where several artificial neural networks are jointly used to solve a problem. In this paper, an automatic pathological diagnosis procedure named Neural Ensemble based Detection (NED) is proposed, which utilizes an artificial neural network ensemble to identify lung cancer cells in the images of the specimens of needle biopsies obtained from the bodies of the subjects to be diagnosed. The ensemble is built on a two-level ensemble architecture. The first-level ensemble is used to judge whether a cell is normal with high confidence where each individual network has only two outputs respectively normal cell or cancer cell. The predictions of those individual networks are combined by a novel method presented in this paper, i.e. full voting which judges a cell to be normal only when all the individual networks judge it is normal. The second-level ensemble is used to deal with the cells that are judged as cancer cells by the first-level ensemble, where each individual network has five outputs respectively adenocarcinoma, squamous cell carcinoma, small cell carcinoma, large cell carcinoma, and normal, among which the former four are different types of lung cancer cells. The predictions of those individual networks are combined by a prevailing method, i.e. plurality voting. Through adopting those techniques, NED achieves not only a high rate of overall identification but also a low rate of false negative identification, i.e. a low rate of judging cancer cells to be normal ones, which is important in saving lives due to reducing missing diagnoses of cancer patients.  
Artificial Neural Networks in Biomedicine %2D Paulo J.G. Lisboa
Since the early 1990s, artificial neural networks play an increasing role in the development of new biomedical systems. In United States, over last decades, the granted biomedical patents that explicitly refer to artificial neural networks in their title, abstract or key references amount to about 50% of the total number of granted biomedical patents with a significant element of computational intelligence. Furthermore, the number of granted biomedical patents that explicitly involve artificial neural networks has quadrupled in recent four years. This observation greatly indicates the growing commercial interest in biomedical products involving artificial neural networks.
Hybrid decision tree.
In this paper, a hybrid learning approach named HDT is proposed. HDT simulates human reasoning by using symbolic leaming to do qualitative analysis and using neural leaming to do subsequent quantitative analysis. It generates the trunk of a binary hybrid decision tree according to the binary information gain ratio criterion in an instance space defined by only original unordered attributes. If unordered attributes cannot further distinguish training examples falling into a leaf node whose diversity is beyond the diversity-threshold, then the node is marked as a dummy node. After all those dummy nodes are marked, a specific feedforward neural network named Fnqc that is trained in an instance space defined by only original ordered attributes is exploited to accomplish the leaming task. Moreover, this paper distinguishes three kinds of incremental learning tasks. Two incremental leaming procedures designed for example-incremental leaming with different storage requirements are provided, which enables HDT to deal gracefully with data sets where new data are frequently appended. Also a hypothesis-driven constructive induction mechanism is provided, which enables HDT to generate compact concept descriptions.
Face recognition with one training image per person.
Abstract: Recently, a method called (PC) 2 A was proposed to deal with face recognition with one training image per person. As an extension of the standard eigenface technique, (PC) 2 A combines linearly each original face image with its corresponding first-order projection into a new face and then performs principal component analysis (PCA) on a set of the newly combined (training) images. It was reported that (PC) 2 A could achieve higher accuracy than the eigenface technique through using 10%-15 % fewer eigenfaces. In this paper, we generalize and further enhance (PC) 2 A along two directions. In the first direction, we combine the original image with its second-order projections as well as its first-order projection in order to acquire more information from the original face, and then similarly apply PCA to such a set of the combined images. In the second direction, instead of combining them, we still regard the projections of each original image as single derived images to augment training image set, and then perform PCA on all the training images available, including the original ones and the derived ones. Experiments on the well-known FERET database show that the enhanced versions of (PC) 2 A are about 1.6 % to 3.5 % more accurate and
Genetic Algorithm based Selective Neural Network Ensemble.
Neural network ensemble is a learning paradigm where several neural networks are jointly used to solve a problem. In this paper, the relationship between the generalization ability of the neural network ensemble and the correlation of the individual neural networks is analyzed, which reveals that ensembling a selective subset of individual networks is superior to ensembling all the individual networks in some cases. Therefore an approach named GASEN is proposed, which trains several individual neural networks and then employs genetic algorithm to select an optimum subset of individual networks to constitute an ensemble. Experimental results show that, comparing with a popular ensemble approach, i.e. averaging all, and a theoretically optimum selective ensemble approach, i.e. enumerating, GASEN has preferable performance in generating ensembles with strong generalization ability in relatively small computational cost. 1
An Approach for Data Filtering Based on Rough Set Theory.

Combining Regression Estimators GA%2DBased Selective Neural Network Ensemble.
Neural network ensemble is a learning paradigm where a collection of neural networks is trained for the same task. In this paper, the relationship between the generalization ability of the neural network ensemble and the correlation of the individual neural networks constituting the ensemble is analyzed in the context of combining neural regression estimators, which reveals that ensembling a selective subset of trained networks is superior to ensembling all the trained networks in some cases. Based on such recognition, an approach named GASEN is proposed. GASEN trains a number of individual neural networks at first. Then it assigns random weights to the individual networks and employs a genetic algorithm to evolve those weights so that they can characterize to some extent the importance of the individual networks in constituting an ensemble. Finally it selects an optimum subset of individual networks based on the evolved weights to make up the ensemble. Experimental results show that, comparing with a popular ensemble approach, i.e. averaging all, and a theoretically optimum selective ensemble approach, i.e. enumerating, GASEN has preferable performance in generating ensembles with strong generalization ability in relatively small computational cost. This paper also analyzes the working mechanism of GASEN from the view of error-ambiguity decomposition, which reveals that GASEN improves generalization ability mainly through reducing the average generalization error of the individual neural networks constituting the ensemble.
Pose Invariant Face Recognition.
In this paper, we describe a novel neural network architecture, which can recognize human faces with any view in a certain viewing angle range (fromy left 30 degrees to right 30 degrees out of plane rotation). Viewspecific eigenface analysis is used as the front-end of the system to extract features, and the neural network ensemble is used for recognition. Experimental results show that the recognition accuracy of our network ensemble is higher than conventional methods such as using a single neural network to recognize faces of a specific view.
A Statistics Based Approach for Extracting Priority Rules from Trained Neural Networks.
In this paper, a statistics based approach named STARE that is designed to extract symbolic rules from trained neural networks is proposed. STAVE deals with continuous attributes in a unique way so that not only different attributes could be discretized to different number of clusters but also unnecessary discretization could be avoided. STAVE introduces statistics to the generation and evaluation of priority rules that have concise appearance. Since it is independent of the network architectures and training algorithms, STAVE could be applied to diversified neural classifiers. Experimental results show that rules extracted via STAVE are comprehensible, compact and accurate.
Using Neural Networks for Fault Diagnosis.
In this paper, a universial Fault Instance Model, which aims to solve problems existing in the present technology of fault diagnosis, such as the lack of universiality, the difficulty in the use of real time system and the dilemma of stability and plasticity, is proposed. The experiment demonstrates that the FANNC used can successfully settles the problems mentioned above by its effectively incremental ability and processing new input patterns via one round learning.
A general neural framework for classification rule mining.
Abstract. Neural network technology has already been applied in a variety of domains with remarkable success. However, it has not been well utilized in data mining and knowledge discovery. In this paper, a general neural framework named NEUCRUM is proposed for classiÞcation rule mining. This paper also presents a possible implementation of NEUCRUM whose key components are a speciÞc neural classiÞer named FANNC and a novel rule extraction approach named STARE.FANNC is used to learn from pre-processed data, in which its fast learning speed and strong generalization ability are quite contributive. STARE is proposed in this paper, which is used to extract comprehensible, compact and accurate symbolic rules from trained neural networks so that the knowledge discovered is explicitly available to decision-makers. Applications show that NEUCRUM and its implementation described in this paper work well in many real domains.
FANNC A Fast Adaptive Neural Network Classifier.
In this paper, a fast adaptive neural network classifier named FANNC is proposed. FANNC exploits the advantages of both adaptive resonance theory and field theory. It needs only one-pass learning, and achieves not only high predictive accuracy but also fast learning speed. Besides, FANNC has incremental learning ability. When new instances are fed, it does not need to retrain the whole training set. Instead, it could learn the knowledge encoded in those instances through slightly adjusting the network topology when necessary, that is, adaptively appending one or two hidden units and corresponding connections to the existing network. This characteristic makes FANNC fit for real-time online learning tasks. Moreover, since the network architecture is adaptively set up, the disadvantage of manually determining the number of hidden units of most feed-forward neural networks is overcome. Benchmark tests show that FANNC is a preferable neural network classifier, which is superior to several other neural algorithms on both predictive accuracy and learning speed.
FANRE A Fast Adaptive Neural Regression Estimator.
In this paper, a fast adaptive neural regression estimator named FANRE is proposed. FANRE exploits the advantages of both Adaptive Resonance Theory and Field Theory while contraposing the Characteristic of regression problems. It achieves not only impressive approximating results but also fast learning speed. Besides, FANRE has incremental learning ability.
Mining Typhoon Knowledge with Neural Networks.
Neural network technology has not been fully utilized in data mining. The reason is two-sided. First, most neural algorithms need long-term training, and could not perform incremental learning. Second, the knowledge learned by neural networks is concealed in a lage amount of connections. In this paper, we develop a neural network method to mine typhoon knowledge through overcoming those two obstacles. Experimental results show that our method could return considerable reward.
